add DS V4.1 Flash on MI355X / 新增 MI355X 配方 - #2962
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
将 MI355X changelog 条目关联至 PR #2962。
Oseltamivir
added
full-sweep-fail-fast
full-sweep-enabled
and removed
full-sweep-fail-fast
labels
|
Claude finished @Oseltamivir's task in 4m 16s —— View job Review of PR #2962
Summary: One blocking issue (inline). The launcher mounts the repo at Everything else checks out statically: Not verified in this session: I could not run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| # Follow upstream AMD defaults for Engram; storage behavior needs verification. | ||
| # The upstream ROCm image is not published yet; runtime validation is pending. | ||
| # https://github.com/vllm-project/recipes/blob/main/models/deepseek-ai/DeepSeek-V4.1-Flash.yaml | ||
| source "$(dirname "$0")/../../benchmark_lib.sh" |
There was a problem hiding this comment.
🔴 BLOCKING: The launcher mounts the repo at /ix for this model, but the recipe sources benchmark_lib.sh without setting INFMAX_CONTAINER_WORKSPACE, which defaults to /workspace (benchmarks/benchmark_lib.sh:2829-2831).
Why it matters: Inside the container AGENTIC_DIR/AIPERF_DIR resolve to /workspace/utils/agentic-benchmark and /workspace/utils/aiperf, which do not exist when the repo is at /ix. install_agentic_deps fails at uv pip install -r "$AGENTIC_DIR/requirements.txt" -e "$AIPERF_DIR", so the run dies before the server starts. Even if that passed, write_agentic_result_json does cd "$INFMAX_CONTAINER_WORKSPACE" and writes the agg JSON to /workspace/$RESULT_FILENAME.json (benchmark_lib.sh:3186-3188), which is not the mounted $GITHUB_WORKSPACE, so the workflow's result check fails. The launcher's RESULT_DIR remap only covers the per-run directory, not the library's workspace root. No launcher exports this variable; the existing /ix recipes (dsv4_fp4_b300_sglang_mtp.sh:11-25) resolve it themselves. The new launcher test doesn't exercise the recipe, so it can't catch this.
Fix: Resolve the repo root from the script location and export it before sourcing the library, mirroring the B300 SGLang recipes:
| source "$(dirname "$0")/../../benchmark_lib.sh" | |
| SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | |
| INFERENCEX_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)" | |
| export INFMAX_CONTAINER_WORKSPACE="${INFMAX_CONTAINER_WORKSPACE:-$INFERENCEX_ROOT}" | |
| source "$INFERENCEX_ROOT/benchmarks/benchmark_lib.sh" |
Alternatively, drop the /ix remap in runners/launch_mi355x-amds.sh:307-316 entirely. The sibling dsv4_fp4_mi355x_vllm_mtp.sh runs from /workspace on this same runner with a vLLM ROCm image, and nothing in the diff shows why this image needs a different mount.
| # DSv4.1 weights live on the persistent shared cache. Mount this recipe | ||
| # outside /workspace so runtime setup does not create directories there. | ||
| CONTAINER_REPO=/workspace | ||
| if [[ "$MODEL" == "deepseek-ai/DeepSeek-V4.1-Flash" ]]; then | ||
| export HF_HUB_CACHE_MOUNT="/it-share/hf-hub-cache/" | ||
| CONTAINER_REPO=/ix | ||
| case "${RESULT_DIR:-}" in | ||
| /workspace/*) export RESULT_DIR="/ix/${RESULT_DIR#/workspace/}" ;; | ||
| esac | ||
| fi |
There was a problem hiding this comment.
🔴 For DeepSeek-V4.1-Flash the repo is mounted at /ix instead of /workspace, but INFMAX_CONTAINER_WORKSPACE (used by benchmark_lib.sh to build AGENTIC_DIR/AIPERF_DIR) is never set, so it still defaults to /workspace. install_agentic_deps(), called directly by the new benchmark script, will look for /workspace/utils/agentic-benchmark/requirements.txt and /workspace/utils/aiperf inside a container where the repo is actually at /ix, failing every run for this recipe. Fix: export INFMAX_CONTAINER_WORKSPACE="$CONTAINER_REPO" whenever CONTAINER_REPO is overridden.
Extended reasoning...
benchmark_lib.sh sets AGENTIC_DIR="${INFMAX_CONTAINER_WORKSPACE:-/workspace}/utils/agentic-benchmark" and AIPERF_DIR="${INFMAX_CONTAINER_WORKSPACE:-/workspace}/utils/aiperf" at source time. launch_mi355x-amds.sh mounts GITHUB_WORKSPACE at /ix for this model (lines 309-316) but only exports HF_HUB_CACHE_MOUNT and RESULT_DIR, never INFMAX_CONTAINER_WORKSPACE. dsv41flash_fp4_mi355x_vllm_mtp.sh sources benchmark_lib.sh then calls install_agentic_deps directly (bypassing any per-script override), which will try uv pip install -r /workspace/utils/agentic-benchmark/requirements.txt -e /workspace/utils/aiperf — but /workspace doesn't exist in this container (the repo is at /ix), so the command fails and the server never starts. This is a new bug introduced by this PR's mount rerouting; the added test (test_slurm_utils.py) only checks the srun mount/workdir/RESULT_DIR strings, not that dependent env vars used inside the container by benchmark_lib.sh are consistent with the new mount point.
Verification: normal. The finding is real and reachable in the merged code. launch_mi355x-amds.sh, for MODEL == deepseek-ai/DeepSeek-V4.1-Flash, remaps the repo mount and workdir away from /workspace to /ix: - line 312: CONTAINER_REPO=/ix - line 329: --container-mounts=$GITHUB_WORKSPACE:$CONTAINER_REPO/,... (repo now at /ix, nothing mounted at /workspace) - line 332: --container-workdir=$CONTAINER_REPO/…
|
|
||
| # Draft: upstream documents this ROCm image, but Docker Hub returned 404 on | ||
| # 2026-09-10. Verify publication before dispatching a sweep. | ||
| dsv41flash-fp4-mi355x-vllm-agentic-dspark: |
There was a problem hiding this comment.
🟡 (optional) New config key dsv41flash-fp4-mi355x-vllm-agentic-dspark starts with the literal string "dsv4" with no separator, so infx/matrix/generate.py's full-sweep --model-prefix dsv4 filter (key.startswith(prefix) at generate.py:1002) now also silently matches this unrelated, GPU-unvalidated draft recipe. Every other existing dsv4-* key has an immediate hyphen after "dsv4", so this is the first accidental prefix collision in the dataset. Fix: match on a full path-segment boundary (e.g. key == prefix or key.startswith(prefix + "-")) in generate.py's filter, or rename the model-prefix to avoid the bare-prefix collision (e.g. dsv4.1flash).
Extended reasoning...
An operator runs full-sweep --model-prefix dsv4 --runner-type mi355x-amds ... (a documented, supported CLI pattern per .github/workflows/README.md) intending to sweep only DeepSeek-V4-Pro (dsv4-* keys). Because generate.py's filter at line ~1002 does key.startswith(prefix) on the raw YAML top-level key rather than a path-segment match, "dsv41flash-fp4-mi355x-vllm-agentic-dspark".startswith("dsv4") is True, so this new key is included in the generated matrix alongside the intended dsv4 keys. That recipe uses image vllm/vllm-openai-rocm:deepseekv41-flash-0909, which the PR description says currently 404s on Docker Hub and has never passed GPU validation, so the unintended job fails the image pull/import step, contaminating what the operator believed was a dsv4-only sweep.
Verification: nit. The collision is real and demonstrable. generate.py:1002 filters full-sweep by key.startswith(prefix) on the raw YAML top-level key, and the PR's new key at configs/amd-master.yaml:1763 (dsv41flash-fp4-mi355x-vllm-agentic-dspark) satisfies "dsv41flash-...".startswith("dsv4"). All 17 existing dsv4 keys are dsv4-... and the distinct model families are… | nit. The mechanism is…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34462826434 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34466680355 |
Oseltamivir
changed the title
将 MI355X changelog 条目关联至 PR #2962。
6a21e76 to
0b27640
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34549022944 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34566727564 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34607428979 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34607428979 |
functionstackx
changed the title
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 7135e65. Configure here.
|
|
||
| # Official upstream ROCm image from the vLLM recipe; MI355X runtime validation is pending. | ||
| dsv41flash-fp4-mi355x-vllm-agentic-dspark: | ||
| image: vllm/vllm-openai-rocm:deepseekv41-flash-0909 |
There was a problem hiding this comment.
Recipe pins missing Docker Hub image
High Severity
This change replaces the verified ECR pin with vllm/vllm-openai-rocm:deepseekv41-flash-0909. That Docker Hub tag already 404'd, is still called unavailable in the PR, and is not present on the ROCm repo. enroot import will fail before any GPU work.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit 7135e65. Configure here.
7135e65 to
bc500f1
Compare
新增 MI355X 的 DS V4.1 Flash AgentX 草案配方：TP4、并发 1–32、原生 DSpark 和 AITER，并补充 launcher、测试及中英文文档。ROCm 镜像发布与 Engram 兼容性仍待验证。
将 MI355X DS V4.1 Flash 固定至 semianalysis_cc_traces_weka_062126，避免使用 256k 语料，并同步更新文档与性能变更记录。
DSv4.1 Flash 吞吐测试使用 thinking 开启、五个草稿 token 对应的黄金合成 AL 3.51，关闭自适应验证；准确率评测保留真实验证。同步中英文文档并追加性能变更记录。
使用指定的 ECR ROCm release 镜像替代不可用的 Docker Hub 标签，保留完整上下文语料与 DSpark 设置，并同步文档及性能变更记录。
为 DS V4.1 Flash 的 /ix 挂载设置 INFMAX_CONTAINER_WORKSPACE，修复 AgentX 依赖与输出路径，并验证共享 helper 的实际路径解析。
Upstream vLLM does not apply @support_torch_compile to
DeepseekV41ForCausalLM, so the default cudagraph_mode=FULL_AND_PIECEWISE
aborts at engine init ("piecewise CUDA graphs unavailable, model is not
torch-compiled and breakable CUDA graph is off"). All concurrencies
failed this way in run 34566727564.
The model targets the breakable cudagraph path -- amd/attention.py
imports eager_break_during_capture -- so enable it explicitly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…Sv4.1 Flash MI355X The upstream vLLM recipe image is now public on Docker Hub (AMD-verified, pushed 2026-09-11). Replace the interim ECR release-repo image and update docs/changelog to match.
The EVAL_ONLY speculative config enabled adaptive verification, which trims verification requests on device. The ROCm DeepseekV4IndexerBackend does not support that, so the eval-only engine refused to start (run 34651830283, c32). Keep real block rejection for evals; throughput settings are unchanged. EVAL_ONLY 的推测解码配置启用了自适应验证，它会在设备端裁剪验证请求；ROCm 的 DeepseekV4IndexerBackend 不支持该操作，导致仅评测引擎拒绝启动（运行 34651830283，c32）。评测仍保留真实块拒绝采样；吞吐设置不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
bc500f1 to
86699be
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34651830283 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34657748450 |
|
/reuse-sweep-run 34657748450 |
# Conflicts: # perf-changelog.yaml
2 participants
Add MI355X TP4 DSv4.1 Flash AgentX at concurrency 1–32, using the full
semianalysis_cc_traces_weka_062126corpus. Five-token DSpark throughput uses golden AL 3.51 (thinking on), with adaptive verification disabled; accuracy evals retain real block verification. Uses upstream AMD AITER settings. Uses the official upstream imagevllm/vllm-openai-rocm:deepseekv41-flash-0909from the vLLM recipe, published to Docker Hub on 2026-09-11 after AMD verification (replaces the interim ECR image). Full sweep enabled; Engram compatibility and GPU runtime validation remain pending. Validation: Bash, exact-key matrix, changelog, launcher/eval tests, and both serving-mode command checks pass. GPU validation pending.新增 MI355X TP4 DSv4.1 Flash AgentX，并发 1–32，使用完整
semianalysis_cc_traces_weka_062126语料。五 token DSpark 吞吐测试采用 thinking 开启时的黄金 AL 3.51，并关闭自适应验证；准确率 eval 保留真实块验证。采用上游 AMD AITER 设置。使用上游官方镜像vllm/vllm-openai-rocm:deepseekv41-flash-0909（经 AMD 验证后于 2026-09-11 发布到 Docker Hub，替代临时 ECR 镜像）。已启用 full sweep；Engram 兼容性和 GPU 运行时验证仍待完成。验证：Bash、精确配置矩阵、changelog、launcher/eval 测试及两种服务模式的命令检查通过；GPU 验证待完成。Note
Medium Risk
Introduces a new cluster AgentX sweep key and launcher mount/workspace overrides for one model; misrouting could break AgentX paths or burn GPU time while runtime validation is still pending.
Overview
Adds a draft MI355X AgentX path for DeepSeek-V4.1-Flash (
dsv41flash-fp4-mi355x-vllm-agentic-dspark): TP4, concurrency 1–32, vLLM on ROCm with five-token DSpark, GPU-resident KV, and the full trace corpussemianalysis_cc_traces_weka_062126.A new agentic benchmark script drives serving with upstream AMD AITER/MoE settings,
VLLM_USE_BREAKABLE_CUDAGRAPH=1, throughput pinned to golden AL 3.51 (synthetic rejection), and adaptive verification disabled on ROCm for both throughput and eval (real block rejection on eval).amd-master.yaml, bilingual MODELS matrix rows, configuration docs, and perf-changelog entries register the key; GPU validation remains pending on the officialvllm/vllm-openai-rocm:deepseekv41-flash-0909image.The MI355X launcher remaps this model to mount the repo at
/ix, setINFMAX_CONTAINER_WORKSPACE, shared HF cache, and rewriteRESULT_DIRoff/workspace. A launcher routing test locks that behavior fordsv41flashvsdsv4.Reviewed by Cursor Bugbot for commit 833855b. Bugbot is set up for automated code reviews on this repo. Configure here.